Inter-Coder Agreement in One-to-Many Classification: Fuzzy Kappa
نویسندگان
چکیده
منابع مشابه
Inter-Coder Agreement in One-to-Many Classification: Fuzzy Kappa.
Content analysis involves classification of textual, visual, or audio data. The inter-coder agreement is estimated by making two or more coders to classify the same data units, with subsequent comparison of their results. The existing methods of agreement estimation, e.g., Cohen's kappa, require that coders place each unit of content into one and only one category (one-to-one coding) from the p...
متن کاملInter-Coder Agreement for Computational Linguistics
This article is a survey of methods for measuring agreement among corpus annotators. It exposes the mathematics and underlying assumptions of agreement coefficients, covering Krippendorff’s alpha as well as Scott’s pi and Cohen’s kappa; discusses the use of coefficients in several annotation tasks; and argues that weighted, alpha-like coefficients, traditionally less used than kappa-like measur...
متن کاملWord-Sense Distinguishability and Inter-Coder Agreement
It. is common in NLP that the categories into which text is classified do not have fully objective definitions. Examples of such categories are lexical distinctions such as part-of-speech tags and wordsense distinctions, sentence level distinctions such as phrase attachment, and discourse level distinct.icms such as topic or speech-act categorization. This p>1per presents an approach to analy?-...
متن کاملSurvey Article: Inter-Coder Agreement for Computational Linguistics
This article is a survey of methods for measuring agreement among corpus annotators. It exposes the mathematics and underlying assumptions of agreement coefficients, covering Krippendorff’s alpha as well as Scott’s pi and Cohen’s kappa; discusses the use of coefficients in several annotation tasks; and argues that weighted, alpha-like coefficients, traditionally less used than kappalike measure...
متن کاملFuzzy kappa for the agreement measure of fuzzy classifications
Fuzzy kappa for the agreement measure of fuzzy classifications Weibei Dou , Yuan Ren, Qian Wu, Su Ruan, Yanping Chen, Daniel Bloyet, JeanMarc Constans Department of Electronic Engineering, Tsinghua University, 100084 Beijing, China GREYC-CNRS UMR 6072, 6 Boulevard Maréchal Juin, 14050 Caen, France cCReSTIC, 9 Rue de Qubec,10026 Troyes, France Imaging Diagnostic Center, Nanfang Hospital Guangzho...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: PLOS ONE
سال: 2016
ISSN: 1932-6203
DOI: 10.1371/journal.pone.0149787